Skip to content

[ray-update-2.56.0] Update llm_batch_inference_vision to Ray 2.56.0#887

Merged
Aydin-ab merged 3 commits into
mainfrom
cursor/template-ray-version-3ea0
Jul 8, 2026
Merged

[ray-update-2.56.0] Update llm_batch_inference_vision to Ray 2.56.0#887
Aydin-ab merged 3 commits into
mainfrom
cursor/template-ray-version-3ea0

Conversation

@svc-template-updater

@svc-template-updater svc-template-updater commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Bump llm_batch_inference_vision to Ray 2.56.0.

Changes

  • BUILD.yaml: cluster_env.image_urianyscale/ray-llm:2.56.0-py312-cu130 (was 2.55.1-py311-cu128). Ray 2.56.0 anyscale/ray-llm only ships py312-cu130, so this also moves the Python/CUDA pair (tag verified on Docker Hub).
  • templates/llm_batch_inference_vision/job.yaml: image_urianyscale/ray-llm:2.56.0-py312-cu130.
  • templates/llm_batch_inference_vision/README.md + README.ipynb: in-doc image_uri example → anyscale/ray-llm:2.56.0-py312-cu130.
  • dependencies/template.depsets.yaml: repoint this template's expand entry build_arg_sets ray2551_py311_cu128ray2560_py312_cu130; PyTorch --indexcu130. The ray2560_py312_cu130 bundle + rayllm_2.56.0_py312_cu130.lock base already existed from build(deps): Ray 2.56.0 image base locks (base-only) #833's follow-up, so only this expand pointer + index flipped.
  • templates/llm_batch_inference_vision/requirements.txt: refresh the comment to reference anyscale/ray-llm:2.56.0 and pin huggingface-hub==1.13.0 to match the new base image (was 0.36.2). numpy==1.26.4, pandas==2.3.3, pyarrow==19.0.1, and datasets==3.6.0 kept unchanged — the 2.56.0 ray-llm image still ships numpy 1.x + pyarrow 19 + scipy compiled against numpy 1.x (verified from the CI package diff), so keeping the numpy-1 stack is what actually lets the layered lock reinstall as a no-op.
  • templates/llm_batch_inference_vision/batch_inference_vision.py + batch_inference_vision_scaled.py + README.ipynb/README.md: migrate ray.data.llm.build_llm_processorbuild_processor. Ray 2.56.0 removed the deprecated build_llm_processor alias in [data][llm] Remove guided_decoding, truncate_prompt_tokens, build_llm_processor ray-project/ray#63569.
  • Regenerated templates/llm_batch_inference_vision/python_depset.lock via ./update_deps.sh --name llm_batch_inference_vision_depset_2.56.0_3.12_cu130.

Fix iterations

Two failure→fix cycles (both agent-fixable, not infra):

  1. Build fix the diagrams on stable diffusion notebook #490ImportError: A module that was compiled using NumPy 1.x cannot be run in NumPy 2.2.6 from scipy.io._sparsetools on first import datasets. The Ray 2.56.0 rayllm deplock declares numpy==2.2.6, but the actual anyscale/ray-llm:2.56.0-py312-cu130 image still ships numpy 1.26.4 + pyarrow 19.0.1 (with scipy compiled against numpy 1.x). Fix: pin the layered lock to the runtime versions (numpy==1.26.4, pyarrow==19.0.1), matching the classic "un-pinned floats" gotcha in references/dependencies.md.
  2. Build [template][deps] Adding lock file for ai-libs-training #494ImportError: cannot import name 'build_llm_processor' from 'ray.data.llm'. Ray 2.56.0 removed the deprecated alias ([data][llm] Remove guided_decoding, truncate_prompt_tokens, build_llm_processor ray-project/ray#63569). Fix: rename to build_processor in the two .py files and in the notebook + regenerated README.

Tests / validation

Publish

BUILDKITE_API_TOKEN provided to this Cloud Agent lacks the write_builds scope, so this run cannot programmatically create_build on tmpl-publish (see .claude/skills/template/references/publish-to-backend.md). After merge, a reviewer should trigger a fresh tmpl-publish build (org_slug=anyscale, pipeline_slug=tmpl-publish, branch=master, message=llm_batch_inference_vision) and drive the manual gates dev → staging → production per that reference. Pipeline: https://buildkite.com/anyscale/tmpl-publish.

Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
@svc-template-updater svc-template-updater added ray-update updating template to a new ray version cursor-cloud Generated by Cursor Cloud agents labels Jul 8, 2026
@svc-template-updater

Copy link
Copy Markdown
Contributor Author

/test-template llm_batch_inference_vision

The 2.56.0 anyscale/ray-llm image ships numpy 1.26.4 + pyarrow 19.0.1
(scipy is compiled against numpy 1.x). Upgrading numpy to 2.2.6 via
the layered lock broke scipy on import — pin the runtime versions so
the reinstall is a no-op and keep the numpy-1 ABI intact.

Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
@svc-template-updater

Copy link
Copy Markdown
Contributor Author

/test-template llm_batch_inference_vision

Ray 2.56.0 removed the deprecated build_llm_processor alias in
ray-project/ray#63569; use the canonical build_processor. Updated:
- batch_inference_vision.py / batch_inference_vision_scaled.py
- README.ipynb + regenerated README.md

Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
@svc-template-updater

Copy link
Copy Markdown
Contributor Author

/test-template llm_batch_inference_vision

@svc-template-updater svc-template-updater marked this pull request as ready for review July 8, 2026 01:10
@svc-template-updater svc-template-updater requested a review from a team as a code owner July 8, 2026 01:10
@Aydin-ab Aydin-ab merged commit 55dc66b into main Jul 8, 2026
5 checks passed
Aydin-ab added a commit that referenced this pull request Jul 9, 2026
…#895)

## Summary
Bump `entity-recognition-with-llms` to Ray 2.56.0.

## Changes
- `BUILD.yaml` `cluster_env.image_uri`:
`anyscale/ray-llm:2.55.1-py311-cu128` →
`anyscale/ray-llm:2.56.0-py312-cu130`.
- `templates/entity-recognition-with-llms/containerfile` `FROM`: same
bump.
- **Forced py311→py312 + cu128→cu130 jump.** `anyscale/ray-llm:2.56.0`
on Docker Hub only publishes `py312-cu130` (no `py311-cu128` variant);
this matches the forced-variant handling used for every other
`ray-llm`-based template on this Ray version (see #884, #887, #889,
#892). Tag verified on Docker Hub.
- No `python_depset.lock` for this template (no entry in
`dependencies/template.depsets.yaml`), so the image bump is the whole
change.

## Tests / validation
- **Local:** `pre-commit run --all-files` — passed. `python
ci/validate_build_yaml.py` — passed.
- **CI:** Buildkite `template-test` build
[#506](https://buildkite.com/anyscale/template-test/builds/506) —
**passed** on commit `dee61081` (~44 min end-to-end; a first attempt was
auto-cancelled at ~13 min into training due to a Buildkite agent
shutdown, and the retry ran the full ~34 min LoRA SFT training cleanly
on the new py312-cu130 base).

## Publish
- `BUILDKITE_API_TOKEN` on this Cursor Cloud agent lacks the
`write_builds` scope required to trigger `tmpl-publish` via API
(`create_build` returns `Your access token doesn't have the write_builds
scope`). **Please trigger the publish manually** after merge —
[tmpl-publish pipeline](https://buildkite.com/anyscale/tmpl-publish),
branch `master`, message `entity-recognition-with-llms`; then unblock
`input-tmpl-name` with `tmpl-name=entity-recognition-with-llms`,
`tmpl-branch=main`, `tmpl-commit=HEAD` per `publish-to-backend.md`.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
Aydin-ab added a commit that referenced this pull request Jul 9, 2026
## Summary
Bump `e2e-rag-deepdive` to Ray 2.56.0.

The Anyscale `ray-llm` image only ships as `py312-cu130` for Ray 2.56.0
(per `dependencies/depsets/rayllm_2.56.0_py312_cu130.lock`), so this
bump also migrates the template's base-image Python (3.11 → 3.12) and
CUDA (12.8 → 13.0) variants alongside the Ray version. Same "forced
py311→py312 + cu128→cu130 jump on the LLM image" path other 2.56.0
`ray-llm` bumps have taken (e.g. #892, #891, #887).

## Changes
- `templates/e2e-rag-deepdive/containerfile`: `FROM
anyscale/ray-llm:2.55.1-py311-cu128` → `2.56.0-py312-cu130` (tag
verified on Docker Hub).
- Rebuilt and pushed the template's custom image:
`us-docker.pkg.dev/anyscale-workspace-templates/workspace-templates/e2e-rag-deepdive:2.56.0`.
- `BUILD.yaml`: `cluster_env.byod.docker_image` and `ray_version` →
`2.56.0`.

No `python_depset.lock` / `template.depsets.yaml` entry for this
template (BYOD-only), so no dependency-lock recompile needed.

## Fix iterations
None — CI passed on the first `/test-template` dispatch.

## Tests / validation
- **Local:** `python3 ci/validate_build_yaml.py --no-network` — passed
(56 entries). `pre-commit run --all-files` — passed.
- **CI:** Buildkite `template-test` build
**[#508](https://buildkite.com/anyscale/template-test/builds/508)** —
**passed** (all 3 jobs green; end-to-end ~45 min, well under the
template's 7200s test budget).

## Publish
The read-only Buildkite token in this Cursor Cloud environment lacks the
`write_builds` scope, so this automation cannot trigger the
`tmpl-publish` build itself (a known limitation across all
`template-updater` runs — see prior PRs). After merging, please trigger
the publish manually per
`.claude/skills/template/references/publish-to-backend.md`:

- Pipeline: <https://buildkite.com/anyscale/tmpl-publish>
- Trigger a new build on `master`.
- On the `input-tmpl-name` gate, enter:
  - `tmpl-name=e2e-rag-deepdive`
  - `tmpl-branch=main`
  - `tmpl-commit=HEAD`
- Wait for `build-template` + `test-template` green, then unblock
`block-publish-dev` → `block-publish-staging` →
`block-publish-production` in sequence.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Aydin Abiar <Aydin-ab@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cursor-cloud Generated by Cursor Cloud agents ray-update updating template to a new ray version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants